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DETAILED ACTION 

This action is a Non-Final Office Action in response to Applicants' RCE filed on 
10/06/2008 and Applicants' Amendment/Remarks filed 09/09/2008. Claims 1,4-5, 8-13, 
16-17, 20-25, 28-29, 32-36, and 44-52, are pending. Claims 1, 13 and 25 are currently 
amended. Claims 2-3, 6-7, 14-15, 18-19, 26-27, 30-31, and 37-43 were previously 
canceled. Effective filing date is 03/15/2000, (Assignee: Ricoh). 



Continued Examination Under 37 CFR 1.114 

A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .1 7(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
10/06/2008 has been entered. 

Claim Rejections - 35 USC §112 

The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and 
process of making and using it, in such full, clear, concise, and exact terms as to enable any 
person skilled in the art to which it pertains, or with which it is most nearly connected, to make 
and use the same and shall set forth the best mode contemplated by the inventor of carrying 
out his invention. 



❖ Claim(s) 1, 4-5, 8-13, 16-17, 20-25, 28-29, 32-36, and 44-52 are rejected under 
35 U.S.C. 112, first paragraph, as failing to comply with the written description 
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requirement. The claim contains subject matter which was not described in the 
Specification in such a way as to reasonably convey to one skilled in the relevant 
art that the inventors, at the time the application was filed, had possession of the 
claimed invention. 

❖ Additionally, Claims Claim(s) 1 , 4-5, 8-13, 16-17, 20-25, 28-29, 32-36, and 44-52 
are rejected under 35 U.S.C. 112, first paragraph, as failing to comply with the 
enablement requirement. The claims contain subject matter which was not 
described in the specification in such a way as to enable one skilled in the art to 
which it pertains, or with which it is most nearly connected, to make and/or use 
the invention. 

> Independent claims 1,13 and 25 recite the limitations "the audio sound 
can be extracted directly from the first bar code without having to access 
resources outside of the first multimedia document", this limitation was not 
in the original disclosure (see the amended portions of claims 1,13 and 25 
in the listing of the claims Pages 2, 5, and 8). 

> There is no mention in the original disclosure of making any "without 
having to access resources outside of the first multimedia document' The 
phrase "without having to access resources outside of the first multimedia 
document' or any variant thereof, is not mentioned in the original 
disclosure of the present application. Similarly, in the context of the 
language recited in Claims 1,13 and 25 (see the claims Pages 2, 5, and 
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it is noted, the specification indicates a method of creating a multimedia 
annotation for a paper document and creating a first multimedia document by combining 
the paper document and the multimedia annotation represented by a first bar code 
encoding the at least one of the audio sound and video clip, wherein the video and/or 
audio are stored on a portable storage device or on a network or server [emphasis 
added], See Applicant's Specification at Pages 9, Line 1 through Pages 10, line 20. 

As recognized by the Examiner, a first multimedia document being created by 
combining the paper document and the multimedia annotation represented by a first bar 
code encoding the audio sound and video clip, wherein the video and/or audio are 
stored on a portable storage device or on a network or server, wherein the video and/or 
video is not stored on the first multimedia document, but rather resided on the a 
portable storage device or on a network or server that is accessible upon request by 
activating the bar code which is encoded and printed onto the first multimedia 
document, see Claims 1,13 and 25 in lighted of the Applicants' Specification at Pages 
9, Line 1 through Pages 10, line 20. 

Accordingly, Claims 1, 4-5, 8-13, 16-17, 20-25, 28-29, 32-36, and 44-52 fail to 
comply with the written description requirement. 

In addition to not mentioning that "the audio sound can be extracted directly from 
the first bar code without having to access resources outside of the first 
multimedia document' is made, the original disclosure does not mention how such 
accessing audio and/or video would be made and use. 
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Thus, Claims -5, 8-13, 16-17, 20-25, 28-29, 32-36, and 44-52 fail to comply with 
the enablement requirement. 

In addition, 

❖ The following is a quotation of the second paragraph of 35 U.S. C. 112: 

The specification shall conclude with one or more claims particularly pointing out and 
distinctly claiming the subject matter which the applicant regards as his invention. 

Claims 4-5, 8-13, 16-17, 20-25, 28-29, 32-36, and 44-52 are rejected under 35 
U.S.C. 1 1 2, second paragraph, as being indefinite for failing to particularly point out and 
distinctly claim the subject matter which applicant regards as the invention. 

Claims 1, 13 and 25 recite the phase "the audio sound can be extracted directly 
from the first bar code without having to access resources outside of the first multimedia 
document", (see the amended portions of claims 1,13 and 25 in the listing of the claims 
Pages 2, 5, and 8). 

This limitation is indefinite because it is unclear the metes and bounds of the 
claimed invention. Since phase "the audio sound can be extracted directly from the first 
bar code without having to access resources outside of the first multimedia document' 
or any variant thereof, is not mentioned in the original disclosure of the present 
application. 

Applicant must amend the limitation to particularly point out and distinctly claim 
the subject matter which the Applicant regards as his invention. 

If the examiner has overlooked the portion of the original disclosure that 
describes this feature of the present invention and how to make and use this feature of 
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the present invention, then Applicant should specifically point it out (by page number 
and line number) in the response to this Office Action. 

In the interest of compact prosecution, the application is further examined against 
the prior art, as stated below, upon the assumption that the applicants may overcome 
the above stated rejections under 35 U.S.C. 112. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as 
set forth in section 102 of this title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole would have been obvious 
at the time the invention was made to a person having ordinary skill in the art to which said 
subject matter pertains. Patentability shall not be negatived by the manner in which the invention 
was made. 

Claims 1, 4, 8, 13, 16, 20, 25, 28, 32 and 44-52 are rejected under 35 
U.S.C. 1 03(a) as being as being unpatentable over Morris et al US005420974A issued 
05/30/1995 (hereinafter Morris), in view of Hoda et al US004831 610 filed 03/03/1987 
(hereinafter Hoda). 

Regarding independent claim 1: Morris teaches: 

a method comprising: creating a multimedia annotation for a paper 
document, the multimedia annotation representing at least an audio and a 
video clip; 
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(See Morris at Fig/ 9A-B and Col. 2, Lines 35-40, discloses scanned-in form, and are 
stored in a mixed object document content architecture envelope (MODCA). Morris 
further discloses, if the box being defined is a check box, then the sub-type is defined as 
audio, video, image, text, or other type object. The resulting directory and the image 
object of the overall scanned-in form are stored in a mixed object document content 
architecture envelope (MODCA)-see Morris at Col. 2 Lines 20-30.) 
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creating a first multimedia document by combining the paper 
document and the multimedia annotation represented by the audio sound 
and video clip, 

(See Morris at Col. 2, Lines 35-40, discloses scanned-in form, are stored in a mixed 
object document content architecture envelope (MODCA). Morris further discloses, if 
the box being defined is a check box, then the sub-type is defined as audio, video, 
image, text, or other type object. The resulting directory and the image object of the 
overall scanned-in form are stored in a mixed object document content architecture 
envelope (MODCA)-see Morris at Col. 2 Lines 20-30.) 



wherein the first multimedia document is generated as a part of 
reproducing the paper document via a document reproduction system, 
wherein the multimedia annotation is captured via an input device of the 
document reproduction system while the paper document is being 
reproduced via the document reproduction system, wherein the captured 
multimedia annotation is encoded, 
(See Morris at Col. 2, Lines 35-40, discloses scanned-in form, are stored in a mixed 
object document content architecture envelope (MODCA). Morris further discloses, if 
the box being defined is a check box, then the sub-type is defined as audio, video, 
image, text, or other type object. The resulting directory and the image object of the 
overall scanned-in form are stored in a mixed object document content architecture 
envelope (MODCA)-see Morris at Col. 2 Lines 20-30. 
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Also see Morris at Col. 2, Lines 25-40, discloses a filled-out hard copy of the form 
is desired to be entered into the image archiving system, the filled-out copy of the 
master form is scanned-in to the system. A special indicium is associated with each 
master form, which uniquely identifies that form and distinguishes it from all other 
master forms created for the system. The system reads the indicium from the scanned- 
in form and then accesses the MODCA envelope the directory for the master form is 
then accessed to obtain the coordinates of each component field on the scanned-in 
form. Then access the stored nonvisual object and play it back for the operator- See 
Morris at the Abstract.) 

wherein the first multimedia document, which when scanned by a 
process, the process causes the printed multimedia annotation to be 
decoded, the at least one of the audio sound and video clip to be extracted 
from the multimedia annotation, 

(See Morris at Col. 2, Lines 25-40, discloses a filled-out hard copy of the form is desired 
to be entered into the image archiving system, the filled-out copy of the master form is 
scanned-in to the system. A special indicium is associated with each master form, which 
uniquely identifies that form and distinguishes it from all other master forms created for 
the system. The system reads the indicium from the scanned-in form and then accesses 
the MODCA envelope the directory for the master form is then accessed to obtain the 
coordinates of each component field on the scanned-in form.) 
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and the extracted audio sound and video clip can be played via a 
multimedia player. 

(See Morris at Col. 3, Lines 35-45, discloses when the check box is selected, the 
multimedia record is accessed from the MODCA envelope, for example, and is played. 
If it is an audio record, then the sound is played for the operator. If it is a video record, 
then a video cameo display window can be presented to the operator on the display 
screen and the video sequence played (i.e. played via a multimedia player). 

In addition, Morris does not expressly teach, but Hoda teaches: 

and multimedia annotation represented by a first bar code, wherein 
the captured multimedia annotation is encoded within the first bar code, 
and wherein the audio sound can be extracted directly from the first bar 
code without having to access resources outside of the first multimedia 
document. 

(See Hoda at Col. 1 Lines 25-40, described a bar code which is printed close to an item 
published on a legible recording medium [dictionaries, catalogs, guidebooks, etc., 
generally contain a number of photographs and other pictures and diagrams relating to 
descriptions, guidance material etc. which appear in the printed text, see Col. 1, Lines 
15-20], recognition is performed of an operating command which is indicated by the 
printed bar code, whereby a data recording medium playback apparatus is controlled 
such as to execute playback of video or audio data relating to that item. Also Hoda 
further discloses an example of a page in guidebook 17. Numeral 18 denotes code data 
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printed on that page, which in this example is in the form of bar code data, (referred to 
in the following simply as bar code), which contains codes respectively expressing a 
media player to playback the video or audio data relating to that item. This is generally 
disclosed at Col. 5, lines 9-27 and at Fig. 2. 
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Therefore, the artisan would have well appreciated that Hoda's method of 
captured video or audio annotation is encoded as a bar code, and can be extracted 
directly from the bar code without having to access resources outside of the first 
multimedia document as cited in independent claim 1 . 

This interpretation is supported by the applicant's disclosure, which is stated," the 
user generates the multimedia annotation on a workstation, records it on a portable 
storage device .... all the available multimedia annotations stored on the portable 
storage device and allow the user to review and select them", See Applicant's 
Specification at Page 9, Lines 4-20.) 

Accordingly, It would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Morris's check box and the sub-type is defined as audio, 
video, image, text, or other type object scanned-in form, are stored in a mixed object 
document content architecture envelope (MODCA), to include a means of utilizing the 
barcode as taught by Hoda, so that, when selected by a user (i.e. scanned document 
with the box is check as taught by Morris ) the barcode would accessing the audio 
sound without having to access resources outside of the first multimedia document 
(Hoda at col.1 , line 25 through col. 2. line 60). 

Regarding independent claim 13, 

Is directed to machine-readable medium providing instructions, which 
when executed by a set of one or more processors to perform the method of 
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claim 1 which cites above, and is similarly rejected under the same rationale, 
Also See Morris at Fig. 1 and also at Col. 5, Lines 10-20. 

Regarding independent claim 25, 

Is directed to a computer system include a bus, a data storage, and a 
processor to perform the method of claim 1 which cites above, and is similarly 
rejected under the same rationale, Also See Morris at Fig. 1 and also at Col. 5, 
Lines 10-20. 

Regarding claims 4, 16, and 28, 

Morris and Hoda teach the method of claims 1,13 and 25 and further comprise: 

wherein a location indicator associated with the multimedia 
annotation is placed on the first multimedia document, wherein the location 
indicator indicates where the multimedia annotation can be retrieved and 
played. 

(See Morris at Col. 3, Lines 35-45, discloses when the check box is selected, the 
multimedia record is accessed from the MODCA envelope, for example, and is played. 
If it is an audio record, then the sound is played for the operator. If it is a video record, 
then a video cameo display window can be presented to the operator on the display 
screen and the video sequence played (i.e. played via a multimedia player). 
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Regarding claims 8, 20, and 32, 

Morris and Hoda teach the method of claims 1,13 and 25 and further comprise: 
creating a second multimedia document by combining the image of 
the paper document and the multimedia annotation; and storing the image 
of the paper document and the multimedia annotation in a storage, wherein 
the second multimedia document is an electronic document associated 
with the first multimedia document which is a physical document. 
(See Morris at Col. 2, Lines 35-40, discloses scanned-in form, are stored in a mixed 
object document content architecture envelope (MODCA). Morris further discloses, if 
the box being defined is a check box, then the sub-type is defined as audio, video, 
image, text, or other type object. The resulting directory and the image object of the 
overall scanned-in form are stored in a mixed object document content architecture 
envelope (MODCA)-see Morris at Col. 2 Lines 20-30. 

Also see Morris at Col. 2, Lines 25-40, discloses a filled-out hard copy of the form 
is desired to be entered into the image archiving system, the filled-out copy of the 
master form is scanned-in to the system. A special indicium is associated with each 
master form, which uniquely identifies that form and distinguishes it from all other 
master forms created for the system. The system reads the indicium from the scanned- 
in form and then accesses the MODCA envelope the directory for the master form is 
then accessed to obtain the coordinates of each component field on the scanned-in 
form. Then access the stored nonvisual object and play it back for the operator- See 
Morris at the Abstract.) 
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generating an image of the paper document, the image of the paper 
document being unconsciously captured via the document reproduction 
system during the reproduction of the paper document without user 
intervention; 

(See Morris at Col. 2, Lines 25-40, discloses a filled-out hard copy of the form is desired 
to be entered into the image archiving system, the filled-out copy of the master form is 
scanned-in to the system. A special indicium is associated with each master form, which 
uniquely identifies that form and distinguishes it from all other master forms created for 
the system. The system reads the indicium from the scanned-in form and then accesses 
the MODCA envelope the directory for the master form is then accessed to obtain the 
coordinates of each component field on the scanned-in form. Then access the stored 
nonvisual object and play it back for the operator- See Morris at the Abstract.) 

Regarding claim 44, Morris teaches: 

wherein the first multimedia document is a physical document 
having the first check box printed thereon, which when scanned by a 
scanning device, causes the first check box to be decoded and the audio 
sound to be extracted from the first check box, and wherein the extracted 
audio sound is capable of being played via an audio player. 

(See Morris at Col. 3, Lines 35-45, discloses when the check box is selected, the 

multimedia record is accessed from the MODCA envelope, for example, and is played. 

If it is an audio record, then the sound is played for the operator. If it is a video record, 
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then a video cameo display window can be presented to the operator on the display 
screen and the video sequence played (i.e. played via a multimedia player). 

In addition, Morris does not expressly teach, but Hoda teaches: 
bar code, 

(See Hoda at Col. 3 Lines 60-65, discloses high density barcodes.) 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Morris's check box and the sub-type is defined as audio, video, 
image, text, or other type object scanned-in form, are stored in a mixed object document 
content architecture envelope (MODCA), to include a means of utilizing the barcode as 
taught by Hoda, so that, so that, when selected by a user (i.e. scanned document with 
the box is check as taught by Morris ) the barcode would accessing the audio sound 
without having to access resources outside of the first multimedia document (Hoda at 
col.1 , line 25 through col. 2. line 60). 

Regarding claim 45, Morris teaches: 

capturing an audio sound of the multimedia annotation from a user 
using a microphone of the input device to annotate the paper document to 
create a multimedia paper document. 

(See Morris at Col. 2, Lines 5-20, discloses sound records, Video records, sequential 
animation sequence records, and the like, such multimedia objects being characterized 
in that their presentation requires duration of time. It is within the scope of the invention 
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that check boxes can also be used to represent visual fields such as other image 
objects, other text objects, other vector graphics objects, etc. Morris at Col. 3 Lines 10- 
15, discloses the steps that allow a user to add new recorded segments of audio, video, 
etc. for any page of data. It is inherently including the use of Microphone and Video 
camera in order to record sound and Video as taught by Morris.) 

Regarding claim 46, Morris teaches: 

wherein the microphone is automatically activated when the user 
selects a reproduction function of the document reproduction system to 
reproduce the paper document. 

(See Morris at Col. 2, Lines 17 -60, discloses sound records, Video records, sequential 
animation sequence records, and the like, such multimedia objects being characterized 
in that their presentation requires duration of time. It is within the scope of the invention 
that check boxes can also be used to represent visual fields such as other image 
objects, other text objects, other vector graphics objects, etc. Morris at Col. 3 Lines 10- 
15, discloses the steps that allow a user to add new recorded segments of audio, video, 
etc. for any page of data. It is inherently including the use of Microphone and Video 
camera in order to record sound and Video as taught by Morris. Also Morris discloses if 
the object is a nonvisual field object, such as a voice object, then the operator is 
prompted to input the voice record. Also, the audio check box corresponding to the 
audio record is highlighted- see Morris at Col. 4 Lines 1-15.) 
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Claim 47, Morris teaches: 

capturing a video clip of the multimedia annotation from a user 

using a video camera of the input device to annotate the paper document 

to create a multimedia paper document. 

(See Morris at Col. 2, Lines 5-20, discloses sound records, Video records, sequential 
animation sequence records, and the like, such multimedia objects being characterized 
in that their presentation requires duration of time. It is within the scope of the invention 
that check boxes can also be used to represent visual fields such as other image 
objects, other text objects, other vector graphics objects, etc. Morris at Col. 3 Lines 10- 
15, discloses the steps that allow a user to add new recorded segments of audio, video, 
etc. for any page of data. It is inherently including the use of Microphone and Video 
camera in order to record sound and Video as taught by Morris.) 

Claim 48, Morris teaches: 

wherein the video camera is automatically activated when the user 
selects a reproduction function of the document reproduction system to 
reproduce the paper document. 

(See Morris at fig. 9 and Col. 9, Line 45 through Col, 10, Line 5, discloses Step 1040 
determines whether a video check box has been marked. In the example shown in FIG. 
9A, the video check box 04 has been marked with the mark X which is reference 60. 
Step 1040 then prompts the operator to input the video record. Step 1042 forms a video 
object content architecture (VDOCA) object 58. Step 1044 stores the object 58 in the 
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MODCA envelope 40, as is shown in FIG. 9B and adds the object name " VIDEO-1" as 
the pointer in the check box 04 line of the directory 42C (Table 5). It then adds the 
object name " VIDEO-1" and the media address " MODCA-VOCA" in the video data line 
of the directory 42C (Table 5). It is inherently including the use of Microphone and Video 
camera in order to record sound and Video as taught by Morris.) 

Claim 49, Morris teaches: 

in response to a request to retrieve a second multimedia document, 
performing a content-based search for the requested multimedia document 
within the storage based on the content of the multimedia annotation 
associated with the requested multimedia document. 
(See Morris at Col. 3, Lines 35-45, discloses when the check box is selected, the 
multimedia record is accessed from the MODCA envelope, for example, and is played. 
If it is an audio record, then the sound is played for the operator. If it is a video record, 
then a video cameo display window can be presented to the operator on the display 
screen and the video sequence played (i.e. played via a multimedia player). 

Claims 50-52, Morris teaches: 

wherein the content-based search is performed by OCR, audio 
speech recognition, or video face recognition techniques on the 
multimedia annotations of the multimedia documents being searched, 
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(See Morris at Col. 2, Lines 35-40, discloses scanned-in form (I.e. OCR), are stored in a 
mixed object document content architecture envelope (MODCA). Morris further 
discloses, if the box being defined is a check box, then the sub-type is defined as audio, 
video, image, text, or other type object. The resulting directory and the image object of 
the overall scanned-in form are stored in a mixed object document content architecture 
envelope (MODCA)-see Morris at Col. 2 Lines 20-30.) 

Claims 5, 9-11, 17, 21-23, 29 and 33-35 are rejected under 35 U.S.C. 103(a) 
as being as being unpatentable over Morris et al US005420974A -issued 05/30/1995 
(hereinafter Morris), in view of Hoda et al US004831 610 filed 03/03/1987 (hereinafter 
Hoda), further in view of Schena et al US006448979B1 filed 01/25/1999 (hereinafter 
Schena). 

Regarding claims 5, 17, and 29, 

Morris and Hoda do not expressly teach, but Schena teaches: 

wherein the location indicator comprises a first Uniform Resource 
Locator (URL), and a second bar code, wherein the first URL is indicated in 
plain text, and wherein the second bar code represents the first URL in an 
encrypted form. 

(Schena col. 4, lines 45-55, discloses the link information corresponding to the provider 
information may include a universal resource locator (URL) (Schena col.1, line 64 
through col. 2. line 40). In addition, the links can be encoded according to provider, for 
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example UPC or ISBN numbers and any code may serve as the alphanumeric 
sequence. 

Also See Schena at Col. 3 Lines 60-65, discloses high density barcodes. 
Under the broadest reasonable interpretation, The Examiner equates the claimed bar 
code as equivalent to the UPC or ISBN numbers and any code as taught by Schena. 
This interpretation is supported by Applicant's Specification, which states "The 
annotation may be in different forms, such as, for example, a bar code containing an 
audio message or a URL indicating a link to a video clip" at page 7 lines 15-18.) 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify schena's user selectable object (i.e. the location indicator comprises 
a first Uniform Resource Locator (URL), and a second bar code, wherein the first URL is 
indicated in plain text, and wherein the second bar code represents the first URL in an 
encrypted form) providing a user with access to the portion of the recorded information 
corresponding to the visual feature (i.e. machine readable barcode), to includes a 
means of selecting by a user onto a check box, the sub-type is defined as audio, video, 
image, text, or other type object scanned-in form, are stored in a mixed object document 
content architecture envelope (MODCA) as taught by Morris and Hoda. One of ordinary 
skill in the art would have been motivated to modify this combination because Schena 
and Morris are from the same field of endeavor of electronic presentation includes 
annotation to presentation material, and provides a predictable result of the bridging the 
gap between the virtual multimedia-based of the Internet word and the physical world of 
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printed media, wherein the annotate media may contain one or more of textual, audio, 
or video information, and the customer premises equipment ("CPE"), serving as the 
receiver, plays multimedia sequence (Schena col.1, lines 5-15, col. 1 line 64 through 
col. 2. line 40). 

Regarding claims 9, 21, and 33, 

Morris and Hoda do not expressly teach, but Schena teaches: 

wherein a multimedia document is represented as a second Uniform 

Resource Locator (URL) printed on the first multimedia document, and 

wherein the image of the paper document and the multimedia annotation is 

accessed with the second URL, 
(Schena col. 4, lines 45-55, discloses the link information corresponding to the provider 
information may include a universal resource locator (URL) (Schena col.1, line 64 
through col. 2. line 40). In addition, the links can be encoded according to provider, for 
example UPC or ISBN numbers and any code may serve as the alphanumeric 
sequence. 

Also See Schena at Col. 3 Lines 60-65, discloses high density barcodes. 
Under the broadest reasonable interpretation, The Examiner equates the claimed bar 
code as equivalent to the UPC or ISBN numbers and any code as taught by Schena. 
This interpretation is supported by Applicant's Specification, which states "The 
annotation may be in different forms, such as, for example, a bar code containing an 
audio message or a URL indicating a link to a video clip" at page 7 lines 15-18.) 
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It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify schena's user selectable object (i.e. the location indicator comprises 
a first Uniform Resource Locator (URL), and a second bar code, wherein the first URL is 
indicated in plain text, and wherein the second bar code represents the first URL in an 
encrypted form) providing a user with access to the portion of the recorded information 
corresponding to the visual feature (i.e. machine readable barcode), to includes a 
means of selecting by a user onto a check box, the sub-type is defined as audio, video, 
image, text, or other type object scanned-in form, are stored in a mixed object document 
content architecture envelope (MODCA) as taught by Morris and Hoda. One of ordinary 
skill in the art would have been motivated to modify this combination because Schena 
and Morris are from the same field of endeavor of electronic presentation includes 
annotation to presentation material, and provides a predictable result of the bridging the 
gap between the virtual multimedia-based of the Internet word and the physical world of 
printed media, wherein the annotate media may contain one or more of textual, audio, 
or video information, and the customer premises equipment ("CPE"), serving as the 
receiver, plays multimedia sequence (Schena col.1, lines 5-15, col. 1 line 64 through 
col. 2. line 40). 

Regarding claims 10, 22, and 34, 

Morris and Hoda do not expressly teach, but Schena teaches: 

wherein a third bar code is used to represent a second URL 
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(Schena col. 4, lines 45-55, discloses the link information corresponding to the provider 
information may include a universal resource locator (URL) (Schena col.1, line 64 
through col. 2. line 40). In addition, the links can be encoded according to provider, for 
example UPC or ISBN numbers and any code may serve as the alphanumeric 
sequence. 

Also See Schena at Col. 3 Lines 60-65, discloses high density barcodes. 
Under the broadest reasonable interpretation, The Examiner equates the claimed bar 
code as equivalent to the UPC or ISBN numbers and any code as taught by Schena. 
This interpretation is supported by Applicant's Specification, which states "The 
annotation may be in different forms, such as, for example, a bar code containing an 
audio message or a URL indicating a link to a video clip" at page 7 lines 15-18.) 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify schena's user selectable object (i.e. the location indicator comprises 
a first Uniform Resource Locator (URL), and a second bar code, wherein the first URL is 
indicated in plain text, and wherein the third bar code represents the second URL in an 
encrypted form) providing a user with access to the portion of the recorded information 
corresponding to the visual feature (i.e. machine readable barcode), to includes a 
means of selecting by a user onto a check box, the sub-type is defined as audio, video, 
image, text, or other type object scanned-in form, are stored in a mixed object document 
content architecture envelope (MODCA) as taught by Morris and Hoda. One of ordinary 
skill in the art would have been motivated to modify this combination because Schena 
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and Morris are from the same field of endeavor of electronic presentation includes 
annotation to presentation material, and provides a predictable result of the bridging the 
gap between the virtual multimedia-based of the Internet word and the physical world of 
printed media, wherein the annotate media may contain one or more of textual, audio, 
or video information, and the customer premises equipment ("CPE"), serving as the 
receiver, plays multimedia sequence (Schena col.1, lines 5-15, col. 1 line 64 through 
col. 2. line 40). 

Regarding claims 11, 23, and 35, Morris teaches: 

wherein the recipient is specified by a user via an interface of the 
document reproduction system when the user reproduces the paper 
document using the document reproduction system. 

(See Morris at Col. 2, Lines 35-40, discloses scanned-in form, are stored in a mixed 
object document content architecture envelope (MODCA). Morris further discloses, if 
the box being defined is a check box, then the sub-type is defined as audio, video, 
image, text, or other type object. The resulting directory and the image object of the 
overall scanned-in form are stored in a mixed object document content architecture 
envelope (MODCA)-see Morris at Col. 2 Lines 20-30.) 
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In addition, Morris and Hoda do not expressly teach, but Schena teaches: 

automatically sending a second multimedia document to a recipient 
by electronic mail as a part of reproducing the paper document via the 
document reproduction system, 

(See Schena at Col. 4 lines 35, discloses the link information was published or located, 
along with message-specific information. The examiner equates the claimed 
automatically sending a second multimedia document to a recipient by electronic mail 
as equivalent to link information was published or located, along with a message as 
taught by Schena.) 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify schena's automatically sending a second multimedia document to a 
recipient by electronic mail as a part of reproducing the paper document, so that, a user 
can accessing a segment of video file which references scanned document as taught by 
Morris and Hoda. One of ordinary skill in the art would have been motivated to modify 
this combination because Schena and Morris and Hoda are from the same field of 
endeavor of electronic presentation includes annotation to presentation material, and 
provides a predictable result of the bridging the gap between the virtual multimedia- 
based of the Internet word and the physical world of printed media, wherein the 
annotate media may contain one or more of textual, audio, or video information, and the 
customer premises equipment ("CPE"), serving as the receiver, plays multimedia 
sequence (Schena col.1, lines 5-15, col. 1 line 64 through col. 2. line 40). 
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Claims 12, 24, and 36 are rejected under 35 U.S.C. 103(a) as being as being 
unpatentable over Morris et al US005420974A issued 05/30/1995 (hereinafter Morris), 
in view of in view of Hoda et al US004831 610 filed 03/03/1987 (hereinafter Hoda), 
further in view of Schena et al US006448979B1 filed 01/25/1999 (hereinafter Schena), 
and further in view of Hallida v et al., US 5,880,740 filed 7/12/1996 (hereinafter 
Halliday). 

Regarding claims 12, 24, and 36, 

Morris, Hoda and Schena do not expressly teach, but Halliday teaches: 

wherein the recipient receives the image of the paper document and 
the multimedia annotation in the form of Multi-purpose Internet Mail 
Extension (MIME). 

(See Halliday at Col. 8 Lines 5-28, discloses MIME.) 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Morris and Hoda teaching of using a scanner capable of receiving 
data such as machine-readable code from a printed medium, so that, when selected by 
a user the barcode would accessing a segment of video file which references scanned 
document as taught by Morris, to includes MIME functionality as taught by Halliday. One 
of ordinary skill in the art would have been motivated to modify this combination provide 
a predictable result of the bridging the gap between the virtual multimedia-based of the 
Internet word and the physical world of printed media, wherein the annotate media may 
contain one or more of textual, audio, or video information, and the customer premises 
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equipment ("CPE"), serving as the receiver, plays multimedia sequence (Schena col.1, 
lines 5-15, col. 1 line 64 through col. 2. line 40), and provides a composite document as 
a MIME attachment to an email message and sent via an conventional email utility, 
such as the popular Eudora, America Online, or Netscape Navigator mail programs (see 
Halliday at Col. 7, Line 39, and Col. 8 Lines 10-15). 

It is noted that any citations to specific, pages, columns, lines, or figures in the 
prior art references and any interpretation of the references should not be considered to 
be limiting in any way. A reference is relevant for all it contains and may be relied upon 
for all that it would have reasonably suggested to one having ordinary skill in the art. 
See, MPEP2123. 

Response to Arguments 

Applicant's arguments with respect to claims 1,4-5, 8-13, 16-17, 20-25, 28-29, 
32-36, and 44-52 filed 09/09/2008 [see the Remarks Pages 12-16], have been 
considered but are moot in view of new ground of rejection presented in this Office 
Action and further view of the followings: 

In addition, "What matters is the objective reach of the claim. If the claim extends 
to what is obvious, it is invalid under § 103." KSR Int'l Co. v. Teleflex, Inc., 127 S. Ct. 
1727, 1742 (2007). To be nonobvious, an improvement must be "more than the 
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predictable use of prior art elements according to their established functions." Id. at 
1740. 

In this case, the Examiner's analysis Morris discloses scanned-in form, are 
stored in a mixed object document content architecture envelope (MODCA), at Col. 3, 
lines 35-45. Morris further discloses at the Abstract, a multimedia document form 
system enables the definition, manipulation, storage and editing of a form which 
contains multimedia objects. For example an audio object will be entered in digitized 
form and stored in association with the form. When the form is then displayed on a 
visual display device, the corresponding check box is highlighted to indicate to the 
operator the presence of a nonvisual presentation object which is stored in association 
with the form. The operator can then select the check box with a mouse, and the system 
will then access the stored nonvisual object and play it back for the operator, See the 
Abstract. 

In view of Schena's machine-readable code on the paper is used to communicate 
corresponding multimedia information when the machine-readable code is read by a 
scanner. The URL encoded in the machine-readable code points to a multimedia file. 
Thus, the machine-readable code on the printed medium annotates the printed medium 
with the referenced multimedia file, thus combining the printed medium with virtual 
multimedia. In combining the multimedia presentation with the printed medium, Schena 
has created a multimedia document of the claimed invention (See Schena the Abstract) 
and also Schena in col. 2 lines 6-8 that the multimedia annotation may contain one or 
more of textual, audio, or video information, and further in view of Halliday's teaching of 
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MIME (Multi-purpose internet mail), as for [claims 12, 24 and 36 only] at Col. 8, Lines 5- 
28. 

As recognized by the Examiner, Morris, Schena, and Halliday do not expressly 
teach the audio sound can be extracted directly from the first bar code without having to 
access resources outside of the first multimedia document as newly amended to claims 
1,13 and 25. On the other hand, in what is fairly characterized as analogous art in 
accordance the above-noted case law, Hoda's method discloses this limitation as cited 
above. 

Accordingly, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Morris's check box and the sub-type is defined as audio, 
video, image, text, or other type object scanned-in form, are stored in a mixed object 
document content architecture envelope (MODCA), to include a means of utilizing the 
barcode as taught by Schena, further include Halliday's teaching of MIME (Multi- 
purpose internet mail) and the audio sound can be extracted directly from the first bar 
code without having to access resources outside of the first multimedia document as 
taught by Hoda, so that, when selected by a user (i.e. scanned document with the box is 
check as taught by Morris ) the barcode would accessing a segment of video file which 
references scanned document, and provides a predictable result of the bridging the gap 
between the virtual multimedia-based of the Internet word and the physical world of 
printed media, wherein the annotate media may contain one or more of textual, audio, 
or video information, and the customer premises equipment ("CPE"), serving as the 
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receiver, plays multimedia sequence (Schena col.1, lines 5-15, col. 1 line 64 through 
col. 2. line 40). 

Accordingly, for at least all the above evidence claims 1, 4-5, 8-11, 13, 16-17, 
20-23, 25, 28-29, 32-35, and 44-52 remain rejected at least at this time, 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Quoc A. Tran whose telephone number is 571-272- 
8664. The examiner can normally be reached on 9AM - 5PM EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Doug Hutton can be reached on 571-272-4137. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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